Multi-step flow fusion: towards accurate and dense correspondences in long video shots

With high quality editing of video shots of arbitrary duration in mind, we focus on this problem: how to construct accurate dense fields of correspondences over extended time periods using series of elementary optical flows. Highly elaborated optical flow estimation algorithms are at hand, and they were applied before for dense tracking by simple accumulation, however with unavoidable position drift. On the other hand, direct longterm point matching is more robust to such deviations, but is very sensitive to ambiguous correspondences. Why not combining the benefits of both approaches? Following this idea, we develop a multi-step flow fusion method that optimally generates a dense long-term displacement field by first merging several candidate estimation paths and then filtering the tracks in the spatio-temporal domain. Our approach permits to handle small and large displacements with improved accuracy and is able to recover a trajectory from temporary occlusions.